Shannon Theory for Compressed Sensing

Compressed sensing is a signal processing technique to encode analog sources by real numbers rather than bits, dealing with efficient recovery of a real vector from the information provided by linear measurements. By leveraging the prior knowledge of the signal structure (e.g., sparsity) and designing efficient non-linear reconstruction algorithms, effective compression is achieved by taking a much smaller number of measurements than the dimension of the original signal. However, partially due to the non-discrete nature of the problem, none of the existing models allow a complete understanding of the theoretical limits of the compressed sensing paradigm. As opposed to the conventional worst-case (Hamming) approach, this thesis presents a statistical (Shannon) study of compressed sensing, where signals are modeled as random processes rather than individual sequences. This framework encompasses more general signal models than sparsity. Focusing on optimal decoders, we investigate the fundamental tradeoff between measurement rate and reconstruction fidelity gauged by error probability and noise sensitivity in the absence and presence of measurement noise, respectively. Optimal phase transition thresholds are determined as a functional of the input distribution and compared to suboptimal thresholds achieved by various popular reconstruction algorithms. In particular, we show that Gaussian sensing matrices incur no penalty on the phase transition threshold of noise sensitivity with respect to optimal nonlinear encoding. Our results also provide a rigorous justification of previous results based on replica heuristics in the weak-noise regime.